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Abstract 

This paper discusses an application of the Genetic 
Algorithm , a parallel and global search technique that 
emulates natural genetic operations. Real application 
problems often require optimization of a large number 
of parameters with high precision. Since the existing 
Genetic Algorithms do not represent the parameter sen- 
sitivities, we have devised a novel scheme of Hierarchi- 
cal Genetic Algorithm to solve complicated engineering 
problems. Using this approach, the higher level GAs 
propose promising search spaces, while the lower level 
GAs search in more detail with additional parameter 
sets. This decreases the complexity of search and uti- 
lizes the computing resources efficiently. This scheme 
has been used to design an autonomous control systems 
for space-based resource processing plants. 


1. Introduction 

Applications of computer technology are expanding 
from pure data processing to information and knowl- 
edge processing which enables Computer-Aided Sys- 
tem Design. Knowledge-based system applications are 
characterized by symbolic processing, nondeterministic 
computation, dynamic execution, high potential for par- 
allel and distributed processing and knowledge manage- 
ment. However, fundamental physical limits of current 
technology have not been overcome for the more sophis- 
ticated computation-intensive problems, such as predic- 
tive modeling and forecasting, design automation, large 
scale, simulation and artificial intelligence. The com- 
bination of technology and economic factors make par- 
allel and distributed computing systems attractive and 
effective for a large variety of intelligent machine appli- 
cations $1 

The emergence of massively parallel computers has 
also fueled a growing interest in problem solving sys- 
tems based on principles of evolution and heredity. One 


class of such evolution strategies is Genetic Algorithms 
^1. The remarkable success demonstrated by Genetic 
Algorithms (GA) in search, optimization and learning 
has substantially increased interest in their potential 
application to modeling, simulation and design of com- 
plex real world systems Such applications include 

identification and calibration in model construction and 
subsequent model-based control synthesis and policy 
optimization. However, complex simulations typically 
require large execution times to evaluate alternatives, 
or in GA terms, to obtain fitness values for newly gen- 
erated chromosomal individuals. Such lengthy simu- 
lations present a major bottleneck to GA application 
since tens, or even hundreds, of individuals may need 
to be evaluated in every generation. Parallel processing 
offers promise of reducing this bottleneck along two mu- 
tually supporting avenues: 1) speeding up the simula- 
tion needed to estimate fitnesses using distributed sim- 
ulation methods ® and 2) parallelizing the evaluation 
and processing of fitness information. Both avenues are 
under active investigation and indeed a computer archi- 
tecture to support their integration has been suggested 
[19] 

Real application problems often require optimization 
of a large number of parameters with high precision. 
These parameters increase the complexity of the search 
problem. In existing approaches, a chromosome rep- 
resenting the parameters does not contain information 
about their sensitivity, even though parameters influ- 
ence the system performance to different degrees. 

We have developed a novel scheme of a Hierarchi- 
cal GA optimizer which executes multiple GA modules 
to solve complicated problems. These GA modules are 
constructed hierarchically and creation/deletion is per- 
formed dynamically based on the performance of each 
module. 

Each GA module deals with a different degree of ab- 
stracted models for evaluation and a different number 
of parameters for optimization. High level GA modules 
usually search for fewer parameters which are more sen- 
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sitive to the system performance. They are looking for 
milestones of promising search region instead of accu- 
rate solutions. The candidate individual selected from 
the high level GA module represents a sub search space 
and they are sent to the lower level GA modules for 
more detail search. The lower level GA takes advan- 
tage of the received information, and employs a greater 
number of parameters for further optimization. 

The solutions found at the lower level GA module 
are reported back to the higher level GA module, if it is 
better than the candidate individual from parent mod- 
ule. This information is used to update the fitness of 
the parent. As the purpose of high level GA module 
is not to find actual solution, the models of this level 
are not necessarily accurate. In order to speed up GA 
search, the high level GA modules access less accurate 
models which can reduce simulation-based fitness eval- 
uation time. The basic concept is that of successive 
approximation provided by a nested sequence of mod- 
els 

2. Hierarchical Genetic Algorithms 
for Complex Problems 

A simulation model of such a complex architecture 
is most naturally formulated as a variable structure 
model A Hierarchical GA is implemented on a self- 
organizing variable structure, where creation/deletion 
of modules are determined by their performance. 

2.1 Brief Review of Asynchronous Genetic 
Algorithm 

The GA (genetic algorithm) is a probabilistic algo- 
rithm which maintains a population of individuals, P(t ) 
= xi(f), . . . , x n (<) for iteration t . Each individual repre- 
sents a potential solution to the problem at hand, and, 
in any evolution program, is implemented as some (pos- 
sibly complex) data structure S . Each solution Xi(t) is 
evaluated to give some measure of its fitness. Then 
new population (iteration t + 1) is formed by selecting 
the more fit individuals (select step). Some members 
of new population undergo transformation (recombine 
step) by means of “genetic” operators to form new so- 
lutions. There are unary transformation m t (mutation 
type), which create new individuals by a small change 
in a single individual (m : S - S) and higher order trans- 
formations Cj (crossover type), which create new indi- 
viduals by combining parts from several (two or more) 
individuals The control parameters for genetic op- 
erators (probability of crossover and mutation) need to 
be carefully selected to achieve acceptable performance 
After some number of generations the program 
converges and is successful if the best individual repre- 
sents the optimum solution. 

We have developed concepts for parallel genetic al- 
gorithms that are especially oriented to simulation- 



Figure 1: The Architecture for Asynchronous Genetic 
Algorithm Simulation 

evaluated individuals on high performance computers. 
We have investigated a class of Asynchronous Genetic 
Algorithms (AG As) which does not need to be syn- 
chronized by generations to create successive popula- 
tions. In a multiprocessor architecture, individuals are 
evaluated concurrently and a central agent updates the 
genetic population continuously as the evaluation re- 
sults become known. The motivation behind such asyn- 
chronous updating is the recognition that not only may 
simulation runs be time consuming, but their comple- 
tion times may be highly variable. This variability is 
quite common in performance measuring simulations 1 . 
When such variability is significant, the barrier synchro- 
nization imposed by conventional GAs can greatly im- 
pede search progress since it requires that processing 
cannot proceed to the next generation until the slowest 
individual in the current population has completed its 
evaluation In contrast, the AG A allows new indi- 
viduals to be tested as soon as both the information and 
the computer resources are available to do so. 

A concern immediately raised in the AGA paradigm 
is that the blending of generations occasioned by such 
asynchronized processing might adversely affect the re- 
combination schemes underlying GA search. Certainly, 
the supporting theory typically limits selection, mating, 
crossover and other operations to members of the same 
generation Fortunately, some results in the litera- 

ture suggest that search time and search success are not 
degraded, at least in application to typical test function 
suites 1^1. We note that such artificial fitness functions 
do not include time dependence that might appropri- 
ately suggest the necessity for generational integrity. 

As shown in figure 1, the processing elements (PE) 
in the asynchronous genetic algorithm can be catego- 
rized as : genetic population PE, evaluation PEs and 
control PE. While the PGA executes serial GAs in the 

1 lt certainly occurs when runs are executed in conditional 
mode where termination depends on pre-established criteria (e.g., 
a run may be terminated as soon as failure to achieve a prescribed 
goal is obvious). However, run time variability may also arise 
when runs span a fixed observation interval (on the model time 
base). This may be due to the variability in workload encoun- 
tered by the simulation engine or in the resources allocated to 
the particular trial 
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multiple processing elements with subpopulations, the 
asynchronous genetic algorithm evaluates a single in- 
dividual in a processing elements ( evaluation PE) at 
a time. The total population is always contained in 
the genetic population PE which executes genetic op- 
erations and updates the population. It also generates 
new individuals whenever it receives request from the 
control PE. 

Message transfer between the genetic population PE 
and the evaluation PE is controlled by the control PE : 
it delivers evaluated individuals to the genetic popula- 
tion PE and requests new individuals for the evaluation 
PE. Thus the evaluation PEs keep evaluating individu- 
als with which the genetic population PE continuously 
updates the population. 

2.2 Resolution Increasing Scheme in the 
Hierarchical Genetic Algorithm 

A binary chromosome, a unique knowledge represen- 
tation scheme of Genetic Algorithms, provides a way 
of controlling the accuracy of parameters. The size of 
the binary code determines the number of points to be 
investigated. As more bits are employed, the search 
points increase dramatically (exponentially). Therefore, 
longer string size may provide accurate parameter val- 
ues, but it also makes the GA to search through a large 
number of points. 

As shown in figure 2, same size of binary code can 
increase accuracy as search space changes. At level 1, 
the original search space is defined by MIN and MAX. 
We employs 3 bits and there are 8 possible search points. 
If a certain point (binary code) is selected, the distance 
between its neighbors becomes a new sub search space. 
Therefore the selected candidate individual at the high 
level GA module has meaning as representation of its 
neighbors (sub search space) rather than an actual value 
itself. The same size of binary code is employed with 
the new search space, which increases the accuracy of 
the parameter value. 

2.3 Expanding Search Parameters in the 
Hierarchical Genetic Algorithm 

The previous section explains how search accuracy is 
controlled by the Hierarchical GA. If the search prob- 
lems involve a large number of parameters, the GA takes 
longer or directs to local minima. The Hierarchical GA 
employs an expanding search parameter scheme. The 
higher level starts to search for a small number of pa- 
rameters. The result obtained at the higher level is sent 
to the lower level GA, where extra parameters are in- 
cluded to the received parameters. The lower level GA 
takes advantage of the received information so that it 
need not search all the parameters from the beginning. 

The expanding parameter scheme in the Hierarchical 
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Figure 2: Resolution Control Scheme in Hierarchical 
Genetic Algorithms 

GA also helps determine the optimal number of param- 
eter sets. Complicated designing problems involve the 
evaluation of a large number of parameters, where the 
type of parameters as well as their appropriate values 
are often unknown. 

2.4 Execution of Multiple GA Modules 
in Parallel 

The selected individuals during the search operation 
from the root AG A create lower level GA modules as 
shown in figure 3. The time taken by the module for 
checking its population and selecting a candidate in- 
dividual may be either deterministic or nondeterminis- 
tic. In this experiment, we choose a variable selection 
interval scheme, in which the time intervals of subse- 
quent checking become larger as the GA search con- 
tinues. This strategy is based on a characteristic of GA 
search, the fitness of population increases faster in early 
search stages and saturates to a certain level. Smaller 
checking intervals enables the GA to choose new candi- 
dates before stagnation. 

When a lower level AG A module selects an individual 
as a candidate for the next level, it also reports fitness 
to the parent AGA module. This feedback information 
updates the fitness of parent individual. But the in- 
dividual structure of the parent and child may not be 
comparable to each other because they represent differ- 
ent parameter sets or different search space. The popu- 
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Figure 3: Creation and Execution of multiple AGA in 
Self-Organizing Hierarchical GA 


lation of child AGA searches through the space which is 
suggested by a candidate individual from parent AGA. 
The fitness of the lower level individual, if better, up- 
dates the fitness of parent individual. If updated, the 
fitness of parent no longer represents an actual fitness 
of the binary code. But, since the fitness is increased by 
a certain amount, its value can not be represented in its 
environment. The individual now has a higher fitness 
and has a greater probability of being selected for the 
next evaluation. 

Figure 4 shows the module components at each 
level. Each module contains a controller which ex- 
ecutes AG As to solve a given problem and select 
candidate individual(s) reported from AGAs. It cre- 
ates/deletes lower level modules and communicates with 
higher/lower level modules. This controller is the in- 
telligent component of module and is implemented by 
a rule-based expert system. The module is defined 
as a class in an object-oriented programming environ- 
ment (Chez-Scheme $]) and the same module struc- 
ture is created by higher level object. This module is 
program-interfaced to an AGA procedure written in C. 
The decision making component is implemented in a 
symbolic processing language, while the numeric com- 
putation procedure is written in C. 

When the module receives a problem, it may expand 
search parameters as well as increase resolution. With 
any given problem, the controller first expands parame- 
ters and sends them to the PARA- AG A for GA search. 
The selected individuals at the PARA- AGA are sent to 
HIGH- AG A to increase resolution by the scheme ex- 
plained above. The fitness of the selected individual 
at the PARA-AGA are also reported to the high level 
HIGH- AG A to update the fitness of parent individual. 
The candidate individuals for the next level are selected 
at the HIGH-AGA where the selected individual is also 



Figure 4: Module Components in the Self-Organizing 
Hierarchical GA 

reported to the PARA-AGA of same module. 

3, Design of Control System using 
Self- Organizing Hierarchical GA Environment 

A working prototype of a plant for producing oxy- 
gen from Martian atmosphere, is constructed at NASA- 
UA Space Engineering Center The purpose is to 
evaluate the best designs and operation parameters for 
the Mars mission. Martian C 02 -rich atmosphere is fil- 
tered and compressed to a temperature and pressure 
suitable for electrocatalysis in a Zirconia-based oxygen 
cell. Design issues include the size of the inlet pipe, 
power requirements of the compressor and design of the 
oxygen cell including: cell configuration, material prop- 
erties, electrical parameters such as operating voltage 
and current density, electrode materials, and method of 
application 

In this experiment, we try design an optimal FLC to 
control the temperature of the oxygen production sys- 
tem. The basic idea of the fuzzy control centers around 
the labeling process, in which the reading of a sensor is 
translated into a label as done by human expert con- 
trollers [12] . With expert supplied membership func- 
tions for this labels, a reading of a sensor can be fuzzified 
and defuzztfied. It is important to note that the transi- 
tion between labels are not abrupt and a given reading 
might belong to several label region. 

The fuzzification and defuzzification processing does 
not need to be sequential. The input signal can be 
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(a) Fuzzy Inference Network 



Figure 5: Fuzzy inference network and fuzzy sub- 
spaces: (PL:Positive Large) (PS: Positive Small) 

(ZE:Zero)(NS:Negative Small) (NL:Negative Large) 

fuzzified/defuzzified simulatenously by matching mem- 
bership functions. Therefore fuzzy control processing 
can be adapted to a parallel neural network structure 
where each neuron represents functions (fuzzy member- 
ship) and links represent the weight of a fuzzy rule. 

Figure 5(a) shows the structure of the Fuzzy Neural 
Net Control System (FNC) and its fuzzy subspace (Fig- 
ure 5(b)) [10]. j n thi S experiment, 5 input membership 
functions are assigned to each input signal and 5 out- 
put membership functions are used to compute fuzzy 
output signal. 

While an earlier Fuzzy Logic Controller [10, 20] was 
implemented in rule-based form ( if-ihen ), the FNC em- 
ploys a parallel inferencing network structure. Due 
to the parallel fuzzification/defuzzification scheme, the 
FNC can improve real-time performance of the control 
system for practical application. 

The performance of the FNC is determined by the 



Figure 6: GA optimization of the FNC module 


OjGas 



Zirconia Tubes 


Figure 7: Oxygen Production System Cluster 

input membership functions of layer 1 which fuzzify the 
input signals and the output membership functions of 
layer 4 which defuzzify normalized firing strengths. A 
membership function is specified by number of param- 
eters. 

In order to find a high performance fuzzy member- 
ship functions without the help of human expertise, 
it is necessary to employ computer-aided optimization. 
Since tuning the membership functions requires ad- 
justing many parameters simultaneously, hill-climbing 
search methods would suffer from the complexity of the 
search space. 

For this reason, a probabilistic optimization method 
utilizing evolution strategies, such as Genetic Algorithm 
(GA), was employed to find optimal membership func- 
tions. Since optimizing multi-parameter problems takes 
a long time, we developed new form of GA which is es- 
pecially oriented to parallel computers that can satisfy 
the real-time constraints of the system. 

Figure 6 shows the interaction of the FNC, simula- 
tion model and GA-optimizer. The FNC operates the 
simulation model, such as heater/cluster model of the 
Mars Oxygen Production System (OPS). 

The OPS, shown as in figure 7, includes Zirconia 
tubes located symmetrically inside a cylinder. A radia- 
tion heater is wrapped around the outer surface. With 
this configuration, the majority of heat transfer between 
the outer surface and the oxygen gas inside the system 
is due to radiation. Applying the one- dimensional heat 
equation with lumped temperature distributions for the 
surface and oxygen temperatures we obtained two first 
order differential equations as provided below. The T p 
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3 Membership Functions 



Figure 8: Individual evaluation procedure:FNC opera- 
tion with heater/cluster thermal model 


represents the pipe temperature and T z is Zirconia tube 
temperature. A variable SW is either 0 or 1 to control 
the heat source (heater). The objective of the FNC is 
to increase the temperature of the Zirconia tubes at a 
constant rate until a goal temperature is reached 

^ = 2.75 x SW - 4.42 x l(r 12 (T 4 - T 4 ) - 8.65 x 
10 -4 (T P - 278.0) 

= 4.42 x 10 -12 (T 4 - T 4 ) 

As shown in figure 5, there are two input signals to 
the FNC e.g., temperature increase error rate (inputl) 
and rate of its error rate (input2). Based on two inputs, 
the FNC produces an output command which controls 
on/off duty cycle of the heater element in the model. As 
shown in figure 5, we employed 5 membership functions 
for each input signal and 5 membership functions for 
the output signal. 

Figure 8 provides detailed procedures of the FNC in- 
tegrated with the GA-optimizer. An individual of a GA 
represents one trial set of fuzzy membership functions. 
The GA optimizer sends a parameter assignment to the 
FNC which determines its fuzzy membership functions. 
The model is reset to its initial conditions (starting tem- 
peratures). The operational specifications such as de- 
sired temperature increase rate and goal temperature 
are set inside the controller. The performance of a 
trial individual fitness is measured as the sum of the 
MSE (Mean Square Error) between actual temperature 
increase rate and desired one and maximum absolute 
value of error of temperature increase rate. 

3.1 Design of the FNC for the Oxygen 
Production System 

Our primary objective is to design optimal fuzzy 
membership functions that perform well with given op- 
erational specifications while utilizing minimal human 
expertise. The controller increases the temperature of 
the cell at a constant rate. 




5 Membership Functions 



Designing an optimal FLC involves the investigation 
of several alternatives, such as type of membership func- 
tions and the number required. A single-level GA starts 
to optimize the FLC based on the assumption that 
a given FLC specification, such as type or number of 
membership functions, is optimal. But real world ap- 
plication problem is often too complicated to determine 
the correct system specification. 

Hierarchical GA solves this problem by changing its 
structure according to the performance of each module 
which employs a different number of fuzzy membership 
functions and parameter resolution. Starting from a 
small number of parameters, it expands search parame- 
ters and their resolution as they create lower levels. The 
lower levels take advantage of information found at the 
upper level AGA module. 

Figure 9 shows how search parameters are expanded 
as Hierarchical GA creates lower level modules in the 
example of designing a FLC. The upper level module 
starts to design the FLC with a small number of mem- 
bership functions. Designing a FLC with fewer mem- 
bership functions is relatively easy compared to a large 
number of membership functions. Even though the up- 
per level need not find the best membership function, it 
does provide some information to the lower level which 
supports the design of an optimal FLC. Since the mem- 
bership functions found at the upper level are optimized 
based on constraints of a small number of parameters, 
the lower level GA modules give small tolerances to the 
received parameters. This is due to the effect of new pa- 
rameters on the old parameters optimized earlier. Fig- 
ure 9 illustrates how to expand membership functions, 
the shade area of membership function represents its 
tolerance. 

As we increase the number of employed fuzzy mem- 
bership functions, the fuzzy rule table must also be ex- 
panded. Figure 10 shows ways of adding more slots to 
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Figure 12: Simulation Results from Self-Organizing Hi- 
erarchical GA to design optimal FLC 

improvement of Hierarchical GA shows that when a 
certain GA module is executed, the fitness increases 
suddenly. The FLC specifications of the module pro- 
vides the best performance among other FLC specifica- 
tions. The population of each module represents differ- 
ent species, because they expresses different number of 
parameters and search spaces. When a certain species 
(the correct one) is created, the performance of the Hi- 
erarchical GA improves in a step like manner. The tem- 
perature profile shown in the figure 12 is that of by the 
suggested optimal FLC in the Hierarchical GA. 

4. Conclusions 

Real world application problems often require opti- 
mization of a large number of parameters with high 
precision. These parameters increase the complexity of 
the search problem. In existing GA, a chromosome rep- 
resenting the parameters does not contain information 
about their sensitivity, even though they influence the 
system performance to different degrees. 

We have devised a novel scheme of Hierarchical Ge- 
netic Algorithms in self-organizing variable structure 


Figure 11: Tree of Various FLC Specifications 

the fuzzy rule tables. Since the fuzzy rules are optimized 
based on constraints of fewer membership functions (for 
example, 3 input A,B, and 3 output membership func- 
tions), the expanded fuzzy rules need to be optimized 
with not only more slots but also some degree of toler- 
ance of suggested rule parameters. 

Figure 11 shows a tree of various specifications of the 
FLC in which the Hierarchical GA searches through op- 
timal design. Hierarchical G A first optimizes fuzzy rules 
which are more sensitive to the FLC performance at the 
root module. The small number of fuzzy membership 
functions with small parameter variance were used in 
order to maximize the sensitivity of fuzzy rules. The 
fuzzy rules found at the root module are sent to lower 
levels, where two different membership function types, 
such as triangular and bell shape are employed. The 
lower levels have wider search ranges in the parameters 
and utilize the fuzzy rule information received from the 
root. A greater number of fuzzy membership functions 
are employed when the lower level GA modules are cre- 
ated. 

Figure 12 shows the simulation results that illustrate 
how the Hierarchical GA investigates various FLC spec- 
ifications to design an optimal controller. The fitness 
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environment for complex real world problems. The de- 
sign of a temperature control system for the oxygen pro- 
duction plant was selected as an experiment. Since con- 
ventional control schemes are limited their functional- 
ity to relatively simple applications , Fuzzy Logic/Neural 
Net control methods are received more attention for the 
sophisticated applications. The parameters embedded 
in the controller need to be optimized for the required 
control performance. 

In this paper, a Hierarchical GA investigates various 
FLC specifications using variable structure simulation. 
More sensitive parameters, such as fuzzy rules, are opti- 
mized before other parameters. Higher level G As search 
for candidate individuals that might contain the opti- 
mum in a given search space. These candidates are sent 
to the lower level to be investigated in greater detail. 
If better solutions are found at a lower level, they are 
reported back to the higher level and incorporated into 
its on-going GA search. The higher level GAs search 
in a sparse space with fewer parameters that influence 
the system performance significantly. In order to reduce 
GA search time, higher levels also utilize less accurate 
models for which simulation-based evaluation time is 
reduced. 

The simulation exhibited interesting search behavior 
: when a good GA module is discovered, the perfor- 
mance increases suddenly. This suggests that not only 
has a good design been found, but that all other design 
frameworks can be eliminated. 
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